• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The present invention relates to an image processing method using an iterative Principal Component Analysis (PCA) reconstruction algorithm capable of obtaining a face image from which glasses are removed from a front face image of a collar. The image processing method comprises the steps of a) receiving a color front face image of RGB containing glasses, b) extracting candidate regions of the eye from the received color front face image, and c) a candidate for the extracted eye. Determining an accurate eye area from among the areas, and normalizing the received front face image to a predetermined size, d) color information included in the received front face image and edges of the spectacle frame. E) extracting the eyeglass frame region using the information, e) performing RGB-HSI conversion on the normalized front face image, and f) normalizing the front face of the H-, S-, and I-converted RGB-HSI transforms. Generating correction images H ', S' and I 'with glasses removed based on each of the images, and g) R' by performing HSI-RGB conversion on the correction images H ', S' and I '. Obtaining G ', B' corrected images, and h) R ', G', B And generating a final face image of the color from which the glasses are removed based on the corrected image.
    公开号:KR20040040286A
    申请号:KR1020030006064
    申请日:2003-01-30
    公开日:2004-05-12
    发明作者:김형곤；안상철；오유화
    申请人:한국과학기술연구원；
    IPC主号:
    专利说明:

    Image processing method for removing glasses from a color face image {IMAGE PROCESSING METHOD FOR REMOVING GLASSES FROM COLOR FACIAL IMAGES}
    [15] TECHNICAL FIELD The present invention relates to the field of image processing, and more particularly, to an image processing method for acquiring an image of removing glasses from a front face input image of a collar including glasses using an iterative Principal Component Analysis (PCA) reconstruction algorithm. will be.
    [16] As information society develops, identification technology for identifying people is becoming important, and biometric technology using human features for personal information protection and identification using computers has been studied. Among the biometric technologies, face recognition technology is evaluated as a convenient and competitive biometric technology because of the advantage of identifying the user in a non-contact manner without requiring the user's special motion or behavior. Such face recognition technology is used in various application fields such as identification, criminal detection, human computer interface (HCI), and access control. As such, there are some problems in the practical use of face recognition technology with a wide range of applications, one of which is that the face image is changed due to wearing glasses.
    [17] Conventionally, as an image processing method for removing glasses from a face image including glasses, an algorithm for extracting glasses using a deformable contour and removing the glasses from the face image using the same, an active appearance model Image processing using a PCA algorithm, an algorithm that can remove small occlusion areas (i.e., face parts covered by the glasses), such as glasses, using a flexible model called an active appearance model Method and the like have been proposed.
    [18] Among them, an image processing method using a PCA algorithm is widely used. The PCA algorithm may be classified into two processes as follows. The first process involves a large number of unspecified sample face images without glasses. (Where N = 1,2, ..., M) is a training process that extracts eigenfaces. Here, a plurality of sample face images May include a face image of the same person or a face image of another person. In the second process, the extracted face images including glasses currently input using the extracted unique faces It is a process of acquiring a reconstructed image without glasses.
    [19] First, multiple sample face images that do not include glasses Referring to the first process of extracting the unique face from the following. First, according to Equation 1, a plurality of sample face images for training Where multiple sample face images Average image of each column represented by one column vector) And a plurality of sample face images as shown in Equation 2 Average image from each Subtract
    [20]
    [21]
    [22] Each sample face image Average video From the difference image Φ N minus, each sample face image using Equation 3 below Find the covariance matrix C for.
    [23]
    [24] Here, A is a matrix consisting of difference images Φ N , and A T is a transpose matrix of A.
    [25] As a result, the average image Eigenvectors can be obtained from the covariance matrix C and eigenvectors (Where k = 1, ..., M). Unique face Since the process of acquiring is sufficiently understood by those skilled in the art, it will not be described herein for the sake of simplicity.
    [26] Next, the face input image including glasses The unique face extracted in advance Glasses-free reconstruction images It can be represented by the process as follows. First, an arbitrary face input image, that is, a face input image including glasses, as shown in Equation 4 below. Average video Each unique face you have already extracted Project on.
    [27]
    [28] Where ω k is the face input image Unique face Represents a weight that can be represented in a space composed of. Reconstruction Is a number of sample face image, through the equation (5) below Unique face extracted from It can be expressed in the form of a weighted sum of.
    [29]
    [30] At this time, the unique face required The number of is also available as M 'which is the total number of unique faces or less.
    [31] In the image processing method of the related art as described above, face images not including glasses, that is, a plurality of sample face images Unique face from When extracted, the extracted unique face Since only the features of the human face are included, the face input image including the glasses currently input based on this. Reconstruct the image In this case, an image of a face with glasses removed may be obtained. However, the reconstructed face image in which glasses are removed by the image processing method according to the related art. Looking in detail, as shown in Figure 1 the face input image Images reconstructed but glasses not completely removed, reconstructed face images It can be found that it contains many errors throughout. Here, the client of FIG. 1 refers to a person included in a training set, and a non-client refers to a person not included in a training set. The people included in the training set have unique faces with facial features extracted from them. Because it is reflected in, it is possible to obtain reconstructed facial images of better quality than those not included in the training set.
    [32] However, reconstructed face images obtained by applying the image processing method of the prior art as described above There are several problems to consider as a complete eyeglass removal image as follows. The first problem is the number of sample face images included in the training set. Unique face extracted from Face input video currently input, based on Reconstructed image with glasses corresponding to When creating a face input video The unique features that were included in it will be lost. The second problem is the face input image currently being input Reconstructed face image with glasses removed when occlusion area is very large This reconstructed face image can contain many errors due to the influence of occlusion. Is not only natural but also face input video It can look like a completely different face image.
    [33] As described above, in the prior art, since the problem with the glasses in the input face input image is assumed to be simply a problem due to the frame, there are many constraints to obtain a good quality face image from which the glasses are removed. There is a problem that is difficult.
    [34] Accordingly, in order to solve the above-described problem, the present invention finds both the occlusion region and the occlusion region in the color face image, such as the region generated by the reflection of light on the spectacle lens as well as the spectacle lens. It is an object of the present invention to provide an image processing method using an iterative PCA reconstruction algorithm, which is similar to a face image of a color including an inputted glasses, and can obtain a high quality color face image from which glasses are removed.
    [35] According to the present invention, an image processing method for acquiring an image of removing glasses from a front face image of a collar including glasses through an image processing system is provided. The image processing method comprises the steps of a) receiving a color front face image of RGB containing glasses, b) extracting candidate regions of the eye from the received color front face image, and c) a candidate for the extracted eye. Determining an accurate eye area from among the areas, and normalizing the received front face image to a predetermined size, d) color information included in the received front face image and edges of the spectacle frame. E) extracting the eyeglass frame region using the information, e) performing RGB-HSI conversion on the normalized front face image, and f) normalizing the front face of the H-, S-, and I-converted RGB-HSI transforms. Generating correction images H ', S' and I 'with glasses removed based on each of the images, and g) R' by performing HSI-RGB conversion on the correction images H ', S' and I '. Acquiring the G 'and B' corrected images; and h) R ', G' and B '. And generating a final face image of the color from which the glasses are removed based on the corrected image.
    [1] 1 is a diagram illustrating face input images including glasses and reconstructed face images from which glasses obtained by an image processing method according to the related art using a Principal Component Analysis (PCA) reconstruction algorithm are removed.
    [2] 2 is a flowchart illustrating a glasses removal process according to an image processing method of the present invention using an iterative PCA reconstruction algorithm.
    [3] 3 is a diagram illustrating an iterative PCA reconstruction algorithm for processing a normalized face image of an I (Intensity) component in accordance with the present invention.
    [4] 4 is a view for explaining a process of extracting an eyeglass frame region from a face input image according to the present invention.
    [5] 5 is a view showing a determination range of an occlusion region based on gray-level of a difference image according to the present invention.
    [6] 6 (a) to 6 (c) are diagrams illustrating skin color areas and areas other than skin color, using color information of a face input image according to the present invention;
    [7] 7 is a diagram illustrating a range of weight values used for correction of a reconstructed face image according to the present invention.
    [8] 8 is a flowchart illustrating an iterative PCA reconstruction algorithm for processing normalized face images of S (Saturation) and H (Hue) components in accordance with the present invention.
    [9] 9 is a diagram illustrating an example of corrected images of an I component from which glasses are obtained by processing images of the I component including glasses according to the image processing method of the present invention.
    [10] FIG. 10 is a diagram illustrating examples of final color face images from which glasses obtained by processing front face images of a collar including glasses according to an image processing method of the present invention; FIG.
    [11] 11 is a schematic structural diagram of an image processing system for performing an image processing method according to the present invention;
    [12] 12 is a diagram illustrating an example of corrected images of an S component from which glasses are obtained by processing images of the S component including glasses according to the image processing method of the present invention.
    [13] FIG. 13 is a diagram illustrating an example of corrected images of an H x vector component from which glasses are removed obtained by processing images of the H x vector component among H images including glasses according to the image processing method of the present invention. FIG.
    [14] 14 is a diagram illustrating an example of corrected images of an H y vector component from which glasses are obtained by processing images of the H y vector component among H images including glasses according to the image processing method of the present invention.
    [36] Now, with reference to the accompanying Figures 2 to 14 will be described in detail the image processing method according to a preferred embodiment of the present invention.
    [37] First, referring to FIG. 11, a schematic configuration diagram of an image processing system for performing an image processing method of the present invention using an iterative Principal Component Analysis (PCA) reconstruction algorithm is illustrated. As illustrated in FIG. 11, the image processing system 1000 includes an image input unit 1002, an image processing unit 1004, an image output unit 1006, and a face database (DB) 2000.
    [38] The image input unit 1002 receives a front face input image of a collar including glasses and transmits the image to the image processing unit 1004. The image input unit 1002 is implemented as a conventional image input device such as a digital camera. Can be. The image processing unit 1004 performs a function of generating a face image of a color from which glasses are removed by performing an image processing method according to the present invention on a front face input image of a color received through the image input unit 1002. It may be implemented as a conventional computing device. The face DB 2000 is under the control of the image processing unit 1004, the front face input image of the color input through the image input unit 1002 and the front face input generated during the image processing performed by the image processing unit 1004. Performs a function of storing intermediate processed images of an image. Also, the face DB 2000 stores sample face images without glasses used in an iterative PCA reconstruction algorithm, which will be described later. The image output unit 1006 performs a function of outputting a face image of a color from which glasses are removed by the image processing unit 1004 and may be implemented as a conventional display device such as a monitor.
    [39] Referring to FIG. 2, a flowchart illustrating a process of an image processing method according to the present invention, which is performed by the image processing system 1000 to remove glasses from a front face input image of a collar including glasses, is illustrated. First, the image processing system 1000 receives a front face input image of a color including glasses (hereinafter, referred to as a color face input image) through the image input unit 1002 in step S202.
    [40] The image processing unit 1004 of the image processing system 1000 may use the color information included in the color face input image in operation S204 to convert a binary GSCD (Generalized Skin Color Distribution) converted image representing the skin color of the face. In operation S206, a binary BWCD (BWCD) transformed image obtained by emphasizing black and white in the face is acquired. The process of obtaining the binarized GSCD transform image and the binarized BWCD transform image from the color face input image may be performed according to a method known in the art.
    [41] The image processing unit 1004 performs morphology filtering in step S208 to have a color different from the skin color, such as an eyebrow, mouth, or hair falling in the color face input image from the binarized GSCD transform image. This is done to find the candidate areas of the eye that are needed to normalize the color face input image. In operation S210, the image processing unit 1004 extracts candidate eye regions using the binarized BWCD transform image and the shape-filtered GSCD transform image, and in step S214, determine the correct eye region among the candidate regions. The color face input image is normalized to a face input image having a constant size. (The normalized color face input image is represented by basic components of red, green, and blue in the RGB (Red; Green; Blue) model.) . In operation S216, the image processing unit 1004 converts the normalized color face input image to RGB-HSI to generate a normalized face input image of each HSI component. As is well known, an image of an RGB model may be converted into an image of a HSI (Hatur; Intensity) model, and likewise, an image of an HSI model may be easily converted into an image of an RGB model. In the present invention, RGB-HSI conversion is performed on the normalized color face input image using Equation 6 below to process the input color face image.
    [42]
    [43] Here, H represents a color component having a value of 0 to 360 degrees, S represents a chroma component of a value of 0 to 1, and I represents a luminance component having a value of 0 to 255.
    [44] In operation S212, the image processing unit 1004 extracts the eyeglass frame using color information of the color face input image and edge information of the eyeglass frame. This will be described in detail with reference to FIG. 4.
    [45] First, the image processing unit 1004 performs an AND operation on a GSCD image obtained by performing shape filtering in step S208 of FIG. 2 and a binary BWCD image obtained in step S206 of FIG. 2, to obtain an image 400. do. The image 400 is an image showing a black region and a white region in the color face input image, and includes an eye and an eyebrow region. Next, the image processing unit 1004 adds an image 100 and an image 400, which are gray-level GSCD converted images acquired in step S204 of FIG. 2, to obtain an image 801. do. This image 801 is an image from which eyes and eyebrows are removed from the image 100. The image processor 1004 generates an image 802 by detecting edges in the image 801 through a well-known Sobel method so that the eyeglass frame included in the color face input image can be more accurately represented. Thereafter, the image processing unit 1004 performs an OR operation on the image inverted from the image 801 and the image 802 to obtain an image 803, and then normalizes the image 803 at step S214. An image 800 (hereinafter, referred to as an eyeglass frame image G 800) is obtained by normalizing to the same size as the face input image and removing all of the eyeglass frame region so that nothing except the eyeglass frame region is included.
    [46] Referring back to FIG. 2, the image processing unit 1004 applies a repetitive PCA reconstruction algorithm to each of the normalized face input images of the H, S, and I components in operation S218, and corrects the H, S, and I components. Next, HSI-RGB conversion of the corrected image of the H, S, and I components obtained in step S220 is performed to obtain the final color face image from which the glasses are removed.
    [47] Referring to FIG. 3, a diagram for describing an iterative PCA reconstruction algorithm in which the image processing unit 1004 processes a normalized face input image of an I component among RGB-HSI converted images of a color face image in step S218 is described. Is shown. First, a normalized face image of I component (hereinafter, normalized image of I component) (500), where i is an index representing a pixel in the image) and reconstructed according to a conventional PCA algorithm. That is, the result images reconstructed by the above Equation 5 are images corresponding to (601) of FIG. 3 (hereinafter, reconstructed image of I component). (Referred to as 601). Then, the normalized image of the I component using the following equation (7) Reconstruction image of 500 and I component Difference images between 601 are calculated to obtain images corresponding to 602 of FIG. 3 (hereinafter, referred to as difference image d (i) 602 of the I component).
    [48]
    [49] As shown, it can be seen that in the difference image d (i) 602 of the I component, the eyeglass frame covering the eyebrows cannot be accurately extracted. As such, the reason why the eyeglass frame covering the eyebrows was not extracted accurately is that the normalized image of the I component Eyeglass frame spanning eyebrows at 500, reconstructed image of I component Normalized image of I component because it is represented by eyebrows with low gray levels at 601 Reconstruction image of 500 and I component This is because even if the gray level intensity difference between 601 is obtained, the difference value is too small. As such, if the eyeglass frame covering the eyebrows is not properly removed, the impression of the person may be different, so the normalized image of the I component is normalized. A result similar to 500 becomes difficult to obtain. Therefore, in order to more accurately find the eyeglass region covering the eyebrow, that is, the occlusion region, the present invention uses the eyeglass frame image G 800 (see FIG. 4) extracted in step S212 of FIG. 2. .
    [50] Images corresponding to 603 of FIG. 3 by stretching the difference image d (i) 602 by reflecting gray level information corresponding to the face in the difference image d (i) 602 of the I component. (Hereinafter, referred to as the difference image D (i) 603 of the I component) is generated, which is a reconstructed image of the difference image d (i) 602 of the I component and the I component as shown in Equation 8 below. (601) The square root of the product of each gray level intensity.
    [51]
    [52] Using the difference image D (i) 603 of the I component has the following advantages. First, the normalized image of the I component Within 500, shadow area caused by glasses among occlusion areas does not have a large difference in gray level intensity from the surrounding area as compared with other occlusion areas. It is difficult to remove the shadow area by the glasses. However, by using the difference image D (i) 603 of the I component, it is possible to emphasize the difference in the gray level intensity for the shadow area caused by the glasses, which can be easily removed, thereby obtaining a natural glasses removal image. There is an advantage. In addition, the normalized image of the I component Eyes or eyebrows contained within (500) are important features that determine the impression of a person. The difference image of I component D (i) 603 is used to normalize the I component in these feature regions. The difference in gray level intensity from 500 can be reduced, so that the normalized image of the I component 500 has the advantage that it can be used as it is.
    [53] In order to include the eyeglass frame image G 800 extracted in step S212 of FIG. 2 described above in the difference image D (i) 603 of the I component, first, the difference image D (i) 603 of the I component is included. There should be a distinction between regions with occlusion and regions without occlusion within. In the difference image D (i) 603 of the I component, the error distribution is much larger in the occlusion region by glasses than in the region without occlusion. Using this error distribution, the region without occlusion and the region with occlusion can be distinguished within a gray level range of 0 to 255 as shown in FIG. 5. At this time, the threshold for dividing the boundary of each region is determined by the following equation (9).
    [54]
    [55] Where T L represents a lower threshold value, T H represents an upper threshold value, D (j) represents an error value in an area without occlusion, and D (k) represents occlusion Error value of the area in which the zone exists.
    [56] The process of calculating the lower threshold T L and the upper threshold T H will be described in more detail. In order to find the region D (j) without occlusion at 500, the result of inverting the binarized GSCD transform image using the color information of the color face input image is the binarized BWCD image (FIG. 6A). OR operation is performed on the converted video (Fig. 6 (b)). Normalized image of I component based on OR position By normalizing to the same size as 500, the image of FIG. 6C is obtained. In the image (c) of FIG. 6, the black area is a region D (j) without occlusion as an area in which the skin color in the face is highlighted. Therefore, the lower threshold value T L is obtained by calculating an average of error values corresponding to regions without occlusion in the difference image D (i) 603 of the I component shown in FIG. 3.
    [57] Meanwhile, the area D (k), which may include occlusion by glasses, corresponds to areas indicated by emphasizing the skin color and other colors in the face, and thus, the area D (k) becomes a white area in the image of FIG. . Therefore, the upper threshold value T H is obtained by calculating an average of error values larger than the previously obtained lower threshold value T L among the errors in the difference image D (i) 603 of the I component.
    [58] At this time, there is always an uncertain region (ie, an indeterminate region) including error values larger than the lower threshold T L but smaller than the upper threshold T H in the difference image D (i) 603 of the I component. The spectacle frame area near the eyebrow contains error values whose difference in gray level intensity is generally less than the upper threshold value T H , so it is not included in the occlusion area. Therefore, the eyeglass frame region G 800 extracted in step S212 of FIG. 2 is used so that the eyeglass frame region near the eyebrows can be included in the region where the gray level intensity difference values are larger than the upper threshold value T H , that is, the occlusion region. do. As shown in Equation 10 below, at values having a gray level smaller than the upper threshold value T H among the error values in the difference image D (i) 603 of the I component, the eyeglass frame region G 800 has a high gray level value. (That is, G (i)), this G (i) value is used. The images acquired through this process become the difference image D '(i) 604 of the I component shown in FIG.
    [59]
    [60] The difference image D '(i) 604 obtained from the above-described processes is a normalized image of the I component. It is used at 500 to remove occlusion by the glasses. The error values in the difference image D '(i) 604 of the I component have gray levels in the range of 0 to 255, and are non-occlusion areas and indeterminate areas by threshold values determined as described above. , Occlusion region. Now, for each region of the difference image D '(i) 604 of the I component, different weights are given according to Equation 11 below.
    [61]
    [62] Here, ω (i) represents the weight for the difference image D '(i) 604 of the I component.
    [63] A weight of 1 is given to an occlusion region having an error value larger than the upper threshold value T H , and a weight of 0 is given to a non occlusion region having an error value smaller than the lower threshold value T L. Here, giving a weight of 0 means that the original input image is not changed. The uncertainty region is weighted between 0.5 and 1. At this time, the lower limit value of 0.5 is determined through experiments, but the present invention is not limited thereto and the normalized image of the I component. A value that can compensate for the unnatural appearance of the image after removing the occlusion by the glasses included in 500 is sufficient. The above process, normalized image of the I component It is for correcting only the part considered to be glasses at 500. As described above, the weights determined according to Equation 11 (see FIG. 7) are used to correct the occlusion region by the glasses in the difference image D '(i) 604 of the I component through Equation 12. do.
    [64]
    [65] here, Is a corrected image 605 of the I component from which the glasses are removed according to the present invention.
    [66] If the weight is 0, the normalized image of the I component is determined as the non-occlusion region. (500) is used as it is. When the weight is 1, it is determined as the occlusion region, and thus sample face images without glasses are used in the first iteration process (t = 0) of the iterative PCA reconstruction algorithm (ie, the first column of FIG. 3). Image of I component obtained from Using 700, a corrected image 605 of the I component is obtained. The reason is that the reconstructed image of the first reconstructed I component 601 normalized image of I component This is because the glasses included in 500 are not completely removed images. Therefore, from the second iteration process (ie, the second column of FIG. 3), the reconstructed image of the I component obtained by reconstructing the correction image 605 of the I component from which the glasses are removed in the first iteration process (t = 0) Remove the glasses with 601.
    [67] If the weight has a value between 0.5 and 1, it is an indeterminate area where it is difficult to reliably determine the presence or absence of occlusion. Average video of 500 and I components Normalized image of 700 or I component Reconstruction image of 500 and I component Correction is made by combining (601). Specifically, in the first iteration process (t = 0) of the iterative PCA reconstruction algorithm, the average image of the I component The normalized image of the I component and the result of multiplying the weight ω by the gray level intensity value at the position corresponding to the uncertain region in the 700 The glasses are removed using a value obtained by multiplying the result obtained by multiplying the gray level intensity value of the corresponding position by “1-ω” at 500. Normalized image of I component from the second iteration And a reconstructed image of the I component by reconstructing the corrected image 605 of the I component obtained in the first process. Use as 601 to calibrate in the same manner as described above.
    [68] In detail, the first column images of FIG. 3 (normalized image of I component) Reconstructed image of I component (except 500) 601 to I component corrected image 605 is a normalized image of the I component Finding glasses part at 500, normalized image of I component Average image of the I component of the glasses part of 500 Images generated through the above-described process using 700 are described. The second column images of FIG. 3 are reconstructed images of the I component reconstructing the corrected image 605 of the I component corrected in the first iteration process (t = 0). Normalized image of I component, using 601 The images are generated through the combination with 500. The same applies to subsequent columns (ie, from the third column). This iterative PCA reconstruction algorithm is repeatedly performed until the difference between the reconstruction images is less than or equal to the predetermined reference value θ as shown in Equation 13 below.
    [69]
    [70] When the repetition is stopped, that is, when the difference between the reconstructed images is less than or equal to the predetermined reference value θ, the corrected face image generated in the last process is the final corrected image of the I component in the lower right of FIG. 605). An example of the final corrected images of the I component with the glasses removed for the front face image of the collar (hereinafter referred to as I ′ image) is shown at the bottom of FIG. 9.
    [71] 8 is a flowchart illustrating an iterative PCA reconstruction algorithm for processing normalized face input images of S and H components obtained in step S216 of FIG. 2 according to the present invention. In order to acquire the glasses removing image of the color, iterative PCA reconstruction algorithm must be performed on the normalized face input images of the S component and the H component as well as the normalized face input images of the I component described with reference to FIG. At this time, unlike in the normalized face input image processing of the I component described with reference to FIG. 3, in the iterative PCA reconstruction algorithm performed on the normalized face input images of the S component and the H component, it is extracted in step S212 of FIG. 2. Glasses frame image G (800) is not used. The reason is that, as described above, the S and H components mean saturation and color for the face input image of the color, respectively, and the occlusion region obtained from the normalized face input image of the S and H components is obtained from the I component. This is because the occlusion region existing in the normalized face input image is different.
    [72] First, a normalized face input image of an S component (hereinafter, referred to as a normalized image of an S component) (Refer to the top image in FIG. 12), an iterative PCA reconstruction algorithm will be described. Normalized image of S component prior to execution of iterative PCA reconstruction algorithm The image values of are stretched to have a range of 0 to 255.
    [73] In operation S802, the image processing unit 1004 of FIG. 11 may normalize an image of an S component including glasses obtained in operation S216 of FIG. 2. A plurality of sample face images stored in face DB 2000 (see FIG. 11) for Average image of the S component previously obtained with And unique face Normalized image of the S component through the above Equation 5 using Reconstructed image of S component without glasses by reconstructing Create
    [74] In operation S804, the image processing unit 1004 may normalize an image of the S component by using Equation 7 described above. And the reconstructed image of the S component generated in step S802. The difference image d (i) of the S component is obtained from. In operation S806, the image processor 1004 obtains the difference image D (i) of the S component reflecting the feature of the face by stretching the difference image d (i) of the S component by using Equation 8 described above.
    [75] In operation S808, the image processing unit 1004 calculates an average of error values of regions without occlusion in the difference image D (i) of the S component by using Equation 9 described above and lower threshold value T L. Acquire. Also, the image processor 1004 obtains an upper threshold T H by calculating an average of errors larger than the lower threshold T L.
    [76] In operation S810, the image processing unit 1004 uses the above equation (10) to weight the occlusion region with a weight of 1 for the difference image D (i) of the S component, 0 for the non-occlusion region, The indeterminate region generates a corrected image of the S component by weighting a value between 0.5 and 1. At this time, as in the normalized face input image processing of the I component, from the second iteration of the iterative PCA reconstruction algorithm, the reconstructed image of the S component is reconstructed by reconstructing the correction image generated in the first iteration process (t = 0). Use as.
    [77] In step S812, the image processing unit 1004 determines whether the difference between the reconstructed images of the S component is less than or equal to the predetermined reference value θ by using Equation 13 described above, and if the predetermined reference value is less than or equal to the repetitive PCA The reconstruction algorithm is stopped, and the generated corrected image is obtained as a final corrected image (hereinafter, referred to as an S ′ image) of the S component from which the glasses are removed. The generated S 'image is shown at the bottom of FIG. 12.
    [78] In the normalized face input image of the H component, the H component is represented by (H x , H y ) T using Equation 14 below, and the normalized face input image of the S component is described with reference to FIG. 8. Is processed according to the PCA reconstruction algorithm.
    [79]
    [80] As is well known, the image value of the H component in the HSI model is expressed in a circular coordinate system that exhibits the same color as 0 degrees and 360 degrees, unlike the image values of the I component or the S component described above. When the repetitive PCA reconstruction algorithm is applied to the image values of the H component, 0 degrees and 360 degrees representing the same color may not be distinguished, and thus the generated corrected image may include unexpected colors. Therefore, in the present invention, so that the image values of the H component in the performance of the iterative PCA algorithm change, and use by using the equation (14) change to the vector values of the image values of the H component (H x, H y) T form. Each of the images of the H x and H y vector components (see the top image of FIGS. 13 and 14) applies an iterative PCA reconstruction algorithm according to the same process as in the normalized face input image of the S component described above. At this time, prior to the application of the iterative PCA reconstruction algorithm according to the present invention, it is used after stretching the image values of H x and H y vector components to be 0 to 255. Glasses obtained by processing images of H x and H y vector components are removed. And Each of the final corrected images of the vector components is shown at the bottom of FIGS. 13 and 14.
    [81] I ', S', which are correction images obtained through the above-described processes, , In order to obtain the final color glasses removal image based on the image, S 'stretched to have a value in the range of 0 to 255 to apply an iterative PCA reconstruction algorithm. , The image value of the image should be reduced as it is. Then, using Equation 15 below, And An H ′ image, which is a final corrected image of the normalized face input image of the H component, satisfying the image values of the vector component is obtained.
    [82]
    [83] Here, since the image values of the new H ' x and H ' y vector components after removing the glasses do not satisfy H '2 x + H ' 2 y = 1, H ' x (H ' 2 x + H '2 y ) Normalize to 1/2 to get H " x , then find a new H 'in the range of 0 to 360 degrees.
    [84] Then, HSI-RGB conversion is performed on the H ', S', and I 'images by using Equation 16 below. On the basis of the image values of the R ', G', and B 'components finally obtained through the HSI-RGB conversion, a natural final color face image with glasses removed as shown in the lower part of FIG. 10 may be obtained. have. Since this is widely known in the art, a detailed description thereof will be omitted.
    [85]
    [86] Here, each of r, g, and b is a normalized value in the range of 0 to 1 satisfying r + g + b = 1. Equations 6 and 16 described above are well known in the art, so a detailed description thereof will be omitted. (See R. C. Gonzalez and R. E. Woods, "Digital Image Processing," Addison-Wesley Publishing Company, 1992).
    [87] As described above, according to the present invention, by finding and correcting an occlusion region to be removed from the front face input image of the collar, it is possible to obtain a high quality natural color face image from which glasses are removed. In this case, the occlusion region includes not only the spectacle frame but also a region generated by light reflection on the spectacle lens and a shadow region generated in the face due to the spectacles. In addition, the image processing method according to the present invention can be used in various ways to solve other occlusion problems, it is effective to increase the recognition efficiency of the automatic face recognition system.
    [88] While the present invention has been described and illustrated by way of preferred embodiments, those skilled in the art will recognize that various modifications and changes can be made without departing from the spirit and scope of the appended claims.
    权利要求:
    Claims (17)
    [1" claim-type="Currently amended] An image processing method for acquiring an image of removing glasses from a front face image of a collar including glasses, the method comprising:
    a) receiving a color front face image of RGB including glasses, wherein each of the RGB elements comprises a red component, a green component, and a blue component included in the received color front face image; ― And,
    b) extracting candidate regions of the eye from the received color front face image;
    c) determining an accurate eye area from among the candidate areas, and normalizing the received color frontal face image to a predetermined size around the determined eye area;
    d) extracting an eyeglass frame region using color information and edge information of the eyeglass frame included in the received color front face image;
    e) performing RGB-HSI conversion on the normalized front face image;
    f) generating glasses H ', S', I 'with glasses removed based on the normalized front face images of the RGB-HSI converted H, S, I components, said H, S, I components Denotes the Hue component, the Saturation component, and the Intensity component, respectively,
    g) obtaining R ', G', and B 'corrected images by performing HSI-RGB conversion on the corrected images H', S ', and I';
    h) generating a final face image of the color from which glasses are removed based on the R ', G', and B 'correction images;
    Including,
    Step f),
    f1) obtaining reconstructed images of H, S and I components reconstructed normalized front face images of the H, S and I components;
    f2) obtaining first differential images of H, S, and I components between the normalized front face images of the H, S, and I components and reconstructed images of the H, S, and I components;
    f3) H, S, by stretching each of the primary images of the H, S, I components based on pixel information included in the primary images of the H, S, and I components. Obtaining second images of the I component,
    f4) determining a threshold for dividing each of the H, S, and I secondary images into an occlusion region, a non-occlusion region, and an indeterminate region;
    f5) acquiring a third image of component I by including the extracted eyeglass frame region in an indeterminate region divided according to the determined threshold value in the second image of component I;
    f6) distinguishing each of the secondary images of the H component and the S component and the third secondary image of the I component according to the determined threshold value, and assigning a weight to each of the divided regions;
    f7) obtaining the corrected images H ', S', and I 'by applying the weight to each of the second and third images of the H and S components and the third and third images of the I component.
    Image processing method comprising a.
    [2" claim-type="Currently amended] The method of claim 1,
    The normalized front face image of the H component is
    (Equation 14)

    It is expressed as an image of the vector component through, and is a correction image with glasses removed based on the image of the H x and H y vector component And An image processing method for obtaining an image of a vector component.
    [3" claim-type="Currently amended] The method of claim 2,
    (Equation 15)

    Using the above And The correction image H 'is obtained from a correction image of a vector component,
    Wherein H "x is the image processing method will be normalized to" a x (H, H 2 + H x, 2 y) 1/2.
    [4" claim-type="Currently amended] The method of claim 1,
    The corrected images H ', S', and I 'are obtained through a Principal Component Analysis (PCA) reconstruction algorithm,
    Step c) is
    c1) acquiring first and second converted images using the color information included in the received color front face image;
    c2) normalizing the received color front face image using the first and second transformed images;
    Image processing method comprising a.
    [5" claim-type="Currently amended] The method of claim 4, wherein
    And the first converted image is a generalized skin color distribution (GSCD) converted image, and the second converted image is a black and white color distribution (BWCD) converted image.
    [6" claim-type="Currently amended] The method of claim 5, wherein
    And the color information is gray-level pixel information.
    [7" claim-type="Currently amended] The method of claim 5, wherein
    The stretching in step f3),
    (Equation 8)

    Is done using
    Where D (i) is the secondary images of the H, S and I components, Are reconstructed images of the H, S and I components generated during the PCA reconstruction algorithm, d (i) are the first order images of the H, S and I components, and i is an index indicating a pixel in each image ( index) image processing method.
    [8" claim-type="Currently amended] The method of claim 5, wherein
    And the occlusion area is an area including the glasses, a shadow area by the glasses in the received color front face image, an error due to light reflection, and the like.
    [9" claim-type="Currently amended] The method of claim 8,
    Step f4),
    Inverting the first converted image;
    ORing the inverted first transformed image and the second transformed image;
    Determining an average of error values in the ORed image as a lower threshold value;
    Determining an average of error values greater than the lower threshold as an upper threshold
    Image processing method comprising a.
    [10" claim-type="Currently amended] The method of claim 9,
    The lower threshold and the upper threshold,
    (Equation 9)

    Is determined using
    Where D (j) represents the error value of the facial skin region corresponding to the non-closure region in the second images of the H, S, and I components, and D (k) represents the error value of the H, S, and I components. And an error value other than the facial skin region corresponding to the occlusion region in the secondary images.
    [11" claim-type="Currently amended] The method of claim 10,
    The indeterminate region in the secondary image of the I component is
    (Equation 10)

    The third image is obtained by including the extracted eyeglass frame region through
    D '(i) is a third order image of the I component and G (i) is a gray-level value of the extracted eyeglass frame region.
    [12" claim-type="Currently amended] The method of claim 11,
    For the third image of the I component,
    (Equation 11)

    By using, an area having an error value exceeding the upper threshold value has a weight of 1, and an area having an error value less than the lower threshold value has a weight value of 0 and an error value between the lower threshold value and the upper threshold value. Areas should be weighted between 0.5 and 1,
    The region having an error value exceeding the upper threshold is an occlusion region, the region having an error value less than the lower threshold is a non-occlusion region, and the error between the lower threshold and the upper threshold. The area having a value is an indeterminate area.
    [13" claim-type="Currently amended] The method of claim 12,
    The occlusion region in the third image of the component I is
    (Equation 12)

    Is corrected by
    here Is the I 'image, ω is the weight, Is the mean image of the I component, Is a reconstructed image of the I component, Is a normalized front face image.
    [14" claim-type="Currently amended] The method of claim 10,
    An area having an error value exceeding the upper threshold in the second images of the H component and the S component has a weight of 1, and an area having an error value less than the lower threshold has a weight of 0 and the lower threshold. An area having an error value between the upper threshold values is weighted between 0.5 and 1,
    The region having an error value exceeding the upper threshold is an occlusion region, the region having an error value less than the lower threshold is a non-occlusion region, and the error between the lower threshold and the upper threshold. The area having a value is an indeterminate area.
    [15" claim-type="Currently amended] The method of claim 14,
    The occlusion region in the second images of the H component and the S component is
    (Equation 12)

    Is corrected by
    here Is the H 'and S' image, ω is the weight, Is the average image of the H component and the S component, Is a reconstructed image of the H component and the S component, Is a normalized front face image.
    [16" claim-type="Currently amended] The method according to claim 13 or 15,
    The image processing method,
    Repeating the steps f1) to f7)
    Include more,
    When performing the repetition step, an image corrected in step f7) is used in place of the normalized front face image in step f1).
    [17" claim-type="Currently amended] The method of claim 16,
    The image processing method,
    Determining whether a difference between the corrected images generated in the repetition step is equal to or less than a predetermined value;
    If the difference is less than or equal to the predetermined value, outputting the corrected image generated in the repetition step as HSI-RGB conversion and outputting the final face image of the color;
    If the difference exceeds the predetermined value, performing the repeating step again
    Image processing method further comprising.
    类似技术:
    公开号 | 公开日 | 专利标题
    US9836643B2|2017-12-05|Image and feature quality for ocular-vascular and facial recognition
    US9311564B2|2016-04-12|Face age-estimation and methods, systems, and software therefor
    US8582896B2|2013-11-12|Separating directional lighting variability in statistical face modelling based on texture space decomposition
    Asadifard et al.2010|Automatic adaptive center of pupil detection using face detection and cdf analysis
    Proenca2009|Iris recognition: On the segmentation of degraded images acquired in the visible wavelength
    JP2017506379A|2017-03-02|System and method for identifying faces in unconstrained media
    Zhou et al.2007|Appearance characterization of linear lambertian objects, generalized photometric stereo, and illumination-invariant face recognition
    WO2014118842A1|2014-08-07|Makeup assistance device, makeup assistance system, makeup assistance method, and makeup assistance program
    Yan et al.2007|Biometric recognition using 3D ear shape
    US8638993B2|2014-01-28|Segmenting human hairs and faces
    Wang et al.2005|Hallucinating face by eigentransformation
    JP4234381B2|2009-03-04|Method and computer program product for locating facial features
    Kose et al.2013|Countermeasure for the protection of face recognition systems against mask attacks
    JP4428737B2|2010-03-10|Red-eye detection method
    EP1677250B1|2012-07-25|Image collation system and image collation method
    US6504942B1|2003-01-07|Method of and apparatus for detecting a face-like region and observer tracking display
    KR100668303B1|2007-01-12|Method for detecting face based on skin color and pattern matching
    US7460693B2|2008-12-02|Method and apparatus for the automatic detection of facial features
    US8768014B2|2014-07-01|System and method for identifying a person with reference to a sclera image
    US8385609B2|2013-02-26|Image segmentation
    US7415152B2|2008-08-19|Method and system for constructing a 3D representation of a face from a 2D representation
    US9747493B2|2017-08-29|Face pose rectification method and apparatus
    US7512255B2|2009-03-31|Multi-modal face recognition
    US7953253B2|2011-05-31|Face detection on mobile devices
    Viola et al.2004|Robust real-time face detection
    同族专利:
    公开号 | 公开日
    KR100461030B1|2004-12-14|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    法律状态:
    2002-10-31|Priority to KR20020066895
    2002-10-31|Priority to KR1020020066895
    2003-01-30|Application filed by 한국과학기술연구원
    2003-05-10|Priority claimed from PCT/KR2003/000927
    2004-05-12|Publication of KR20040040286A
    2004-12-14|Application granted
    2004-12-14|Publication of KR100461030B1
    优先权:
    申请号 | 申请日 | 专利标题
    KR20020066895|2002-10-31|
    KR1020020066895|2002-10-31|
    [返回顶部]